Existing market surveillance systems provide approaches to monitor and control securities markets. Typically, the market surveillance systems provide a comprehensive real-time database of trading activity, as well as a structure to analyse and process data for suspected market abuse activities.
Traditionally, every transaction performed during a trading day at a market place, such as a security exchange, has on a daily basis at market closing been compressed and stored in a data file in order to make historical transactions manageable and accessible. However, this prior art approach introduces several problems when trying to detect market abuse activities using historical data. For example, when analysing past and present trading activities of a particular participant over a certain period of time, all trading data files within the period must be decompressed and the files searched for every single transaction relating to the participant. With the number of transactions passing through the market surveillance systems reaching levels of 700 000 transactions per second and being on the rise, analysis of data and detection of abusive patterns and activities in historical data, and in particular analysis of current data in real time, is a very cumbersome task.
The increased number of transactions passing through the market surveillance systems results in an ever increasing amount of data to analyse, in order to detect market abuse. The approach in the art of having to decompress great amounts of data all at once and subsequently undertaking vast search and analysis operations makes detection of certain patterns in the recorded difficult if not unfeasible.